Supplementary Material: Structured Transforms for Small-Footprint Deep Learning
نویسنده
چکیده
Proof of Proposition 1.1. The first two properties can be directly verified from the definition. The third property which will turn out to be crucial follows since applying Zf n times cycles the vector back to its original form but with all entries scaled by f . The fourth property follows becauses Zf−1 cancels the downward shift-and-scale action of Zf . The fifth property can be verified by observing the shifting/reversing actions of the left and right hand side on an arbitrary vector.
منابع مشابه
Structured Transforms for Small-Footprint Deep Learning
We consider the task of building compact deep learning pipelines suitable for deployment on storage and power constrained mobile devices. We propose a unified framework to learn a broad family of structured parameter matrices that are characterized by the notion of low displacement rank. Our structured transforms admit fast function and gradient evaluation, and span a rich range of parameter sh...
متن کاملSupplementary Material: Proximal Deep Structured Models
In this supplementary material we first show the analogy between other proximal methods and our proposed deep structured model, including proximal gradient method and alternating direction method of multipliers. After that, we provide more quantitive results on the three experiments. 1 More Proximal Algorithms Examples Let us the consider the problem we defined in Eq. 1 in our main submission. ...
متن کاملLearning Invariance with Compact Transforms
The problem of building machine learning models that admit efficient representations and also capture an appropriate inductive bias for the domain has recently attracted significant interest. Existing work for compressing deep learning pipelines has explored classes of structured matrices that exhibit forms of shift-invariance akin to convolutions. We leverage the displacement rank framework to...
متن کاملMoDL: Model Based Deep Learning Architecture for Inverse Problems
We introduce a model-based image reconstruction framework with a convolution neural network (CNN) based regularization prior. While CNN based image recovery methods are ideally suited for inverse problems with a convolutional structure, deep learning architectures for problems that do not have a convolutional structure (e.g., parallel MRI, structured illumination microscopy) are less obvious. T...
متن کاملDeep Residual Learning for Small-Footprint Keyword Spotting
We explore the application of deep residual learning and dilated convolutions to the keyword spotting task, using the recently-released Google Speech Commands Dataset as our benchmark. Our best residual network (ResNet) implementation significantly outperforms Google’s previous convolutional neural networks in terms of accuracy. By varying model depth and width, we can achieve compact models th...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2015